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ABSTRACT 



Determining how to monitor and enhance school effectiveness 
is one of the major concerns in current education reforms in different areas 
of the world. This paper describes the existing conceptual and practical 
possibilities and dilemmas in developing a framework for monitoring school 
effectiveness. The paper identifies the different types of school 
effectiveness, which include technical/economic, human/social, political, 
cultural, and educational effectiveness at a variety of different 
levels- -individual , institutional, community, societal, and international. 

The paper describes the eight management models for school effectiveness (the 
goal model, system- input model, process model, satisfaction model, legitimacy 
model, ineffectiveness model, organizational learning model, and 
total -quality-management model) . Depending on the types of school 
effectiveness and the models of school management, various conceptual 
possibilities and dilemmas exist in monitoring school effectiveness. The 
paradigm for monitoring should therefore be shifted from a simplistic 
conception to a multi-type and multi-model conception. The paper proposes a 
matrix based on the choice of indicators, the purpose of assessment, and the 
type of evaluation. It also identifies practical dilemmas involved in 
monitoring school effectiveness: (1) balancing between internal development 

and external accountability; (2) balancing between school self-evaluation and 
external evaluation; (3) choosing an appropriate combination of indicators; 

(4) deciding who selects the effectiveness indicators and performance 
standards; and (5) managing the differences between the school-site level and 
the system level. Three tables and three figures are included. (Contains 60 
references.) (LMI) 
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Monitoring School Effectiveness: 
Conceptual and Practical Possibilities and Dilemmas 
in Developing A Framework 



( abstract ) 

How to monitor and enhance school effectiveness is one of the major concerns in current 
education reforms in different areas of the world. From the advances in school effectiveness research, 
this paper aims to illustrate the existing conceptual and practical possibilities and dilemmas in 
developing a framework for monitoring school effectiveness. 

From the multiplicity of school functions in the new century, there may be different types of 
school effectiveness including technical/economic, human/social, political, cultural, and educational 
effectiveness at the individual, institutions, community, society, and international levels. There are 
also eight management models for school effectiveness (including the goal model, system- 
input model, process model, satisfaction model, legitimacy model, ineffectiveness model, 
organizational learning model and total quality management model) and therefore their 
cognate indicators for monitoring are different. According to the types of school 
effectiveness and the models of school management, various conceptual possibilities and 
dilemmas may exist in monitoring school effectiveness. The paradigm for monitoring should be 
shifted from the simplistic conception to a multi-type and multi-model conception. 

Based on the choice of indicators, the purpose of monitoring, and the type of 
evaluation, the paper further proposes a matrix for exploring the critical practical issues, 
dilemmas, and possibilities in monitoring school effectiveness. Implications are drawn from 
these conceptual and practical possibilities and dilemmas for developing a framework for 
monitoring school effectiveness at different levels. Hopefully, they can benefit the current 
educational reforms in the Region and other parts of the world. 
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Conceptual/Practical Possibilities 

Introduction 

Various types of education reforms had started all over the world since the eighties, 
and have become important trends in the nineties. The major reform directions include 
implementing school-based management, establishing systems for quality assurance and 
accountability, emphasizing the application of strategic management and development 
planning, identifying and promoting characteristics of effective schools, and applying total 
quality management in schools ( Caldwell & Spinks, 1988, 1992; Cheng, 1996; Dempster, 
Sachs, Distant, Logan & Tom, 1993; Greenwood & Gaunt, 1994; Hargreaves & Hopkins, 
1991; Mortimore, 1993; Murgatroyd & Morgan, 1993; Murphy & Beck, 1995; Scheerens, 
1992). Among the reform activities, how to monitor and enhance school effectiveness for 
improving education quality has always been the core issue. The aim of this paper is to 
illustrate the existing conceptual and practical possibilities and dilemmas in developing a 
framework for monitoring school effectiveness. It is hoped that the discussion and 
implications can contribute to enhancement of school effectiveness and implementation of 
educational reforms. 



Conceptual Possibilities & Dilemmas: From Multiple School Functions 

To different people, the definition of school effectiveness may be very different. Also, 
school effectiveness is often confused with the term "school efficiency". The critical 
elements of effectiveness conceptualization such as "what criteria", "whose criteria", 
"effective for whom", "who to define", "how to evaluate", "when to evaluate", and "under 
what environmental constraints" are often problematic because there seems no standard 
elements accepted by all concerned constituencies for evaluation (Cheng, 1993a ). 
Particularly, a school is an organization in a changing and complicated social context, 
bounded with limited resources and involving multiple constituencies such as education 
authorities, school administrators, teachers, students, parents, taxpayers, educators, and the 
public. Therefore the people concerned have different interpretations on school functions and 
goals: some regard the short-term effects as important, the others emphasize the long-term 
function; some stress the function on social integration, the others pay attention to personal 
growth. In such a social context, understanding school effectiveness is quite difficult without 
discussing about school functions. To different functions or goals, schools may have different 
performance and effectiveness. For example, some schools may be good at helping students’ 
personal development but some may be excellent in producing competent technicians for the 
needs of the community. Therefore, it is necessary to clarify the aims and functions of 
schools before we can discuss what is school effectiveness and how to evaluate and monitor 
it. 



Multiple School Effectiveness 




Cheng (1996a) provided a comprehensive analysis of multiplicity of school functions. 
From the education aims of schools in USA and Hong Kong (Averch etal., 1974; The 
National Education Goals Panel, 1992; Education & Manpower Branch, 1993), school are 
expected to have multi functions involving individual, institutional, community, national and 
international levels. In addition to education of students, schools serve other implicit or 
explicit functions at different levels in the society according to both Functionalism or the 
Conflict Theory in sociology (Blackledge & Hunt, 1985; Cheng, 1995b). For example, 
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Functionalism suggests that school education can facilitate social mobility and social change 
but the Conflict Theory argues that school education reproduces class structure and 
maintains class inequality at the society level. Based on the commonly espoused education 
goals, organizational studies and development studies (e g., Bolman & Deal, 1991; Cameron 
& Whetten, 1981, 1983; Cheng, 1993 a; Blackledge & Hunt, 1985; Beare & Slaughter, 1993; 
Cheng, 1995b), Cheng ( 1996a) classify the potential school functions into five types: 
technical/economic functions, human/social functions, political functions, cultural 
functions, and educational functions. Accordingly, there are five types of school 
effectiveness, depending on the extent to which schools can perform these functions at 
different levels, as shown in Table 1. 

Technical/Economic School Effectiveness. It refers to the extent to which the 
school can contribute to the technical or economic developments and needs of the 
individual, the institution, the local community, the society, and the international community. 
At the individual level, effective schools can help students to acquire knowledge and skills 
necessary to survive and compete in a modem society or a competitive economy, and 
provide staff job training and opportunity. At the institutional level, effective schools are 
service organizations providing quality service; also they serve as a life place or work place 
of society for clients, employers and all those concerned. At the community and society 
levels, effective schools serve the economic or instrumental needs of the local community, 
supply quality labor forces to the economic system, modify or shape economic behaviors of 
students (future customers and citizens) (McMahon, 1987), and contribute to the 
development and stability of the manpower structure of the economy (Hinchilffe, 1987). At 
the international level, effective school education supplies the high quality forces necessary in 
international competitions, economic cooperation, earth protection, and technology and 
information exchange. 

Human/Social School Effectiveness. It refers to the extent to which the school can 
contribute to human developments and social relationships at different levels of the society. 
As indicated in nearly all formal education goals, at the individual level effective schools help 
students to develop themselves psychologically, socially, and physically, and help them 
develop their potential as fully as possible. At the institutional level, a school is a social entity 
or social system composed of different human relationships. The quality of social climate and 
relationships in it often determines the quality of work life and learning life for teachers and 
students. Therefore an effective school can provide an environment of quality. At the 
community and society levels, according to the perspective of Functionalism effective 
schools serve the social needs or functions of the local community, support social integration 
of multiple and diverse constituencies of society, facilitate social mobility within the existing 
class structure, reinforce social equality for all people of different backgrounds, select and 
allocate competent people to appropriate roles and positions, and contribute to social change 
and development in the long run (Cheng, 1995b). Due to the growing global consciousness 
(Beare & Slaughter, 1993), effective schools are expected to play an important role in 
preparing students for international harmony, social cooperation, global human relationship, 
and elimination of national, regional, racial, and gender biases at the international level such 
that both the local community and the international community can benefit in the long run. 

Political School Effectiveness. It refers to the extent to which the school can 
contribute to the political developments at different levels of society. At the individual level, 
effective schools can help students to develop positive civic attitudes and skills to exercise 
the rights and responsibilities of citizenship. At the institutional level, effective schools act as 
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a place for systematically socializing students into a set of political norms, values and beliefs, 
or for critically discussing and reflecting on the existing political events. An effective school 
often become a political coalition of teachers, parents, and students that can contribute to the 
stability of the political power structure. At the community and society levels, effective 
schools play an important role to serve the political needs of the local community, legitimize 
the authority of the existing government, maintain the stability of political structure, promote 
awareness and movement of democracy, and facilitate the planned political developments and 
changes (Thomas, 1983). The growing awareness of international dependence reinforces the 
need for the contribution of school education to international understanding, global common 
interest, international coalitions, peace movements against war, and elimination of conflicts 
between regions and nations. It seems that schools’ political effectiveness should also be 
important at the international level for the long term benefit of the world. 

Cultural School Effectiveness. It refers to the extent to which the school can 
contribute to the cultural transmission and development at different levels of society. At the 
individual level, effective schools help students to develop their creativity and aesthetic 
awareness and to be socialized with the successful norms, values, and beliefs of society. At 
the institutional level, effective schools act as a place for systematic cultural transmission to 
and reproduction of the next generation, cultural integration among the multiple and diverse 
constituencies, and cultural re-vitalization from the outdated poor traditions. At the 
community and society levels, effective schools often serve as a cultural unit carrying the 
explicit norms and expectations of the local community, transmit all the important values and 
artifacts of the society to students, integrate the diverse sub-cultures from different 
background, and revitalize the strengths of the existing culture such that the society or the 
nation can reduce internal conflicts and wastage and build up a unifying force for national 
benefits. At the international level, effective schools can encourage appreciation of cultural 
diversity and acceptance of different norms, traditions, values, and beliefs in different 
countries and regions, and finally contribute to the development of global culture through 
integration of different cultures. 

Education School Effectiveness. It refers to the extent to which the school can 
contribute to the development and maintenance of education at the different levels of 
society. Traditionally, education is often perceived as only a means for achieving the 
economic, social, political, and cultural values and goals. Due to the rapid development and 
change in nearly every aspect of the world, people begin to accept education in itself as an 
important value or goal. Education represents learning and development. Like economics, 
politics, culture, and social relationship, education becomes a necessary component of our 
life particularly in an era of great change and transformation. The content, system, and 
structure of education need to be developed and maintained. At the individual level, effective 
schools help students to learn how to learn and help teachers to learn how to teach and 
develop professionally. At the institutional level, effective schools serve as a place for 
systematic learning, teaching, and disseminating knowledge, and as a center for 
systematically experimenting and implementing educational changes and developments. At 
the community and society levels, effective schools provide service for different educational 
needs of the local community, facilitate developments of education professions and education 
structures, disseminate knowledge and information to the next generation, and contribute to 
the formation of a learning society. In order to encourage mutual understanding among 
nations and build up “a global family” for the younger generation, effective schools can 
contribute to the development of global education and international education exchange and 
cooperation. 
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Long-Term and Short-Term School Effectiveness 

Taking the time span into consideration, school effectiveness may be further divided 
into two types: long-term effectiveness and short-term effectiveness. Long-term 
effectiveness refers to the contribution or effect of schools that happens and continues in a 
long time (e.g., more than a few years). These types of school effectiveness are often very 
important even though they may not be so obvious to people’s perception. Short-term 
effectiveness refers to the contribution or effect of schools that occurs explicitly in a short 
time (e.g., a few months or less than a few years). In general, for each type of the 
technical/economic, human/social, political, cultural, educational school effectiveness, there 
may exist both long-term and short-term effectiveness even though short-term effectiveness 
may be often more easily identified and commonly emphasized at the individual level or the 
institutional level. 

School Effectiveness and School Efficiency . 

Based on Lockheed (1988), we may differentiate school effectiveness from school 
efficiency in the following way (Cheng, 1993 a): 

When the discussion is mainly in terms of non-monetary inputs or processes (e.g., 
number of textbooks, classroom organization, professional training of teachers, teaching 
strategy, learning arrangements, etc.), the comparison of output function to non-monetary 
input (or process) may be called "School Effectiveness” . If the discussion is mainly in terms 
of monetary input (e.g., 1000 dollars input per student, cost of books, salary, opportunity 
cost, etc.), then the comparison between school output function and monetary input may be 
called " School Efficiency ". With the consideration of the 5 types of school effectiveness at 
five levels, school efficiency may be similarly classified into 25 categories including 
technical/economic efficiency, human/social efficiency, political efficiency, cultural efficiency, 
and educational efficiency at the individual, institutional, community, society, and 
international levels. This classification helps to clarify what kind of effectiveness is under 
discussion. 
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relationships. The findings may be useful to understand how a certain schoc 
effect can be perpetuated from one level to other levels. 

3. The multi-type and single level strategy: This strategy puts its focus on mon 
than one type of school effectiveness and their inter-relationships (e.g., to stud' 
sdiool s technical effectiveness, political effectiveness and ’ cultura 
effectiveness, and their inter-relationships) at a particular level (e.g. at individua 
evel). This strategy can provide the opportunities to understand whether ont 
type of school effectiveness is congruent or conflicting with other types anc 

identify what approach would be used to manage the different school functions 
and enhance their effectiveness. 

4. The multi-type and multi-level strategy: The focus of this strategy include* 
multi-categories and multi-levels of school effectiveness. Obviously it is more 
comprehensive and sophisticated than the above strategies. It c’an provide 
opportunities to understand how different categories of school effectiveness are 
related and how school effectiveness varies across different levels. The findings 
may have more powerful and sophisticated implications for educational reforms 
school management, and educational practice to ensure congruence between 
sdiool functions across and maximize multiple categories of school 
effectiveness. Of course, more resources and time will be needed and the 
research design will be more complicated for this strategy. 

5. The including time frame strategy: Since time frame can be included in 

monitoring school effectiveness, short-term and long-term consideration can be 
added into the above four strategies. In other words, each of the above 
strategies can be further divided into short-term and long-term considerations, 
ror example, the single-cell and single level strategy. 

6. The efficiency consideration strategy: If monitoring school effectiveness 

“ vo V?? the conslder ation of monetary input, the concept of school efficiency 
should be more convenient and therefore, strategies 1 to 4 can be used in terms 
or school efficiency in monitoring. 
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Figure 1 

Conceptual Possibilities from the School Effectiveness Typology 
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Conceptual Dilemmas from Multiple Effectiveness 

Different Constituencies. To different people or constituencies, the expectations 
of school effectiveness are often different. Some people may be more concerned with the 
technical and economic effectiveness but others with the political effectiveness. Some 
people may pay attention to the effectiveness at the individual levels but others may focus 
more on the functions at the community level or society level. Even though in the past 
years school effectiveness at the international level might not attract much attention, there 
seems to be a growing concern about it in recent years (Beare & Slaughter, 1993). 

Different Disciplines. To different academic disciplines, the emphasis on types 
and levels of school effectiveness may be different too. For example, school effectiveness at 
the individual level may receive more attention in educational psychology. For sociology of 
education, school effectiveness at the society level, particularly those related to social 
mobility, equality and class stratification, may attract more concern. Obviously, economics 
of education often focuses on the economic effectiveness at different levels. In the field of 
school management or organizational behavior, school effectiveness at the institutional 
level are inevitably the major topic of study. Obviously different academic disciplines have 
different foci when monitoring school effectiveness. 

Drawbacks from Narrow Conceptions. Based on different beliefs and emphases 
on the types of school effectiveness, different strategies or policies are usually proposed for 
school management and improvement. Traditionally, people often focus narrowly on a few 
categories of school effectiveness such as technical effectiveness and social effectiveness at 
the individual levels and ignore the other categories. The ignorance of a wide spectrum of 
school effectiveness inevitably sets a tight limit to policy-making and management effort for 
school improvement, and the situation that attend to one and neglect the other will often 
happen. Therefore, it is not a surprise that an educational reform policy mainly based on the 
beliefs about technical effectiveness at the individual level cannot improve the cultural or 
social effectiveness at the individual level or other levels. 

Complicated Relationship between Different Types of School Effectiveness. It 
is important to point out that the relationship between the five types of school effectiveness, 
between the five levels of school effectiveness, and even between effectiveness and 
efficiency may be very complicated, and not necessarily positive. A school's high technical 
effectiveness at the individual level does not necessarily promise high technical effectiveness 
or social effectiveness at the society level, although people often assume the existence of 
such a positive relationship (Grosin, 1994). For example, the success of some technical 
training in school does not imply high productivity for the society if these skills learned in 
school are found to be outdated (or useless) in students' later career life. Furthermore, 
many studies from radical perspectives challenge the traditional belief of schools' "positive" 
functions on social equalization by pointing out that schools do not promote social equality, 
they carry on the inequality of social class on the contrary (Blackledge & Hunt, 1985; 
Cheng, 1995b). Therefore even though schools have higher technical effectiveness at the 
institutional level, they do not necessarily have the effectiveness of promoting social 
equality as expected. To a great extent, the relationship between technical effectiveness and 
social effectiveness or cultural effectiveness is very controversial in the field of sociology of 
education. Also, the relationship between technical effectiveness and technical efficiency 
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may not be simple. It is often a hot topic for research in the field of economics of education 
(Cheng & Ng, 1992; Psacharopoulos, 1987). 

In general, a school high in a few of the 25 categories ( 5 types and 5 levels) of 
effectiveness may not be high, too, in the other categories. Furthermore, enhancement of 
one type of school effectiveness does not necessarily promise increase in the other four 
types. Similarly, increasing school effectiveness at one level does not certainly result in 
improvement of effectiveness at the other levels. This is a critical issue and dilemma in 
current implementation of monitoring school effectiveness. More research is needed to 
explore this issue if we want to understand the complicated relationship between different 
types of school effectiveness ( Cheng, 1996b). 

Conceptual Possibilities & Dilemmas: From Multi-Models 

Even though there are five types of school effectiveness at five different levels, 
models used by school practitioners to conceptualize, manage and achieve school 
effectiveness at the school-based level (organizational level) may be very different, 
including the goal model, resource-input model, process model, satisfaction model, 
legitimacy model, organizational model, ineffectiveness model, and total management 
model (Cheng, 1996a). The basic characteristics of the models are summarized in Table 2. 
Each model represents a conceptual possibility for monitoring school effectiveness and it 
has a different set of school-based indicators of effectiveness. 
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Table 2 . . Models of School Effectiveness at the Site-Level 





Conception of 
School Effectiveness 


Conditions for 
Model Usefulness 


Evaluation Indicators/ 
Key Areas (e.g.) 


Goal Model 


• Achievement of stated 
goals 


• Goals are clear, 
consensual, time-bound, 

& measurable; Resources 
are sufficient 


• Objectives listed in the 
school/program plans, 
e.g. achievements 


Resource-Input 

Model 


• Achievement of 
needed resources & 
inputs 


• There is a clear 
relationship between 
inputs & outputs; 
Resources are scarce 


• Resources procured, e.g. 
quality of student intake, 
facilities, financial 
support, etc. 


Process Model 


• Smooth & “healthy” 
internal process 


• There is a clear 
relationship between 
process & outcome 


• Leadership, 
communication, 
participation, 
coordination, social 
interaction, etc. 


Satisfaction 

Model 


• Satisfaction of all 
powerful 
constituencies 


• The demands of the 
constituencies are 
compatible & cannot be 
ignored 


• Satisfaction of Education 
Authorities, management 
board, administrators, 
teachers, parents, 
students, etc. 


Legitimacy 

Model 


• Successful legitimate 
or marketing activities 
for school survival 


• The survival & demise 
among schools must be 
assessed 


• Public relations, 
marketing, public image, 
reputation, status in the 
community, 
accountability, etc. 


Ineffectiveness 

Model 


• Absence of 
characteristics of 
ineffectiveness in 
school 


• There is no consensual 
criteria of effectiveness 
but strategies for school 
improvement are needed 


• Existing conflicts, 
dysfunctions, difficulties, 
defects, weaknesses, etc. 


Organizational 

Learning 

Model 


• Adaptation to 
environmental changes 
& internal barriers 


• Schools are new or 
changing; the 
environmental changes 
cannot be ignored 


• Awareness of external 
needs & changes, internal 
process monitoring, 
program evaluation, 
development planning 


Total Quality 
Management 
Model 


• Total management of 
internal people & 
process to meet 
strategic 

constituencies’ needs 


• The constituencies’ needs 
are compatible; the 
technology & resource are 
available for total 
management 


• Leadership, people 
management, strategic 
planning, process 
management, quality 
results, constituencies’ 
satisfaction, impact on 
society, etc. 




15 



13 



The Goal Model 



APEC 1997 
Conceptual/Practical Possibilities 



The goal model is very often used in evaluating school performance or studying 
school effectiveness. The model assumes that there are clearly stated and generally 
accepted goals for measuring school effectiveness, and that a school is effective if it 
can accomplish its stated goals with given inputs. This model is useful if the school 
outcomes are clear and the effectiveness criteria commonly accepted by all involved 
constituencies are available. The indicators of school effectiveness are often objectives 
listed in school plans and program plans, particularly those related to quality of 
learning and teaching environment, and academic achievements in the public 
examinations, etc. The usefulness of the model is often limited because it depends on 
the clear, measurable, time-bound, and all accepted goals that are often impossible. For 
example, teachers may be more concerned with students’ development of character 
and personality but parents more concerned with students’ examination achievements. 
Employers may emphasize those job-related attitudes and skills of students and policy- 
makers may be more concerned with school contribution to political stability or 
economic growth. It would be important to include a comprehensive set of school 
goals and objectives when the goal model is used to assess school effectiveness. But 
given the limited resources, it is often very difficult for schools to achieve multiple 
goals in a short time (Cameron, 1978; Hall, 1987). Inevitably, dilemmas exist in 
maximizing effectiveness on the multiple goals with limited resources. 

The Resource-Input Model 

Due to the existing pressure of different expectations of multiple powerful 
constituencies, schools often need to pursue multiple but often inconsistent goals. 
Resources become the critical element in school functioning. The resource-input model 
assumes that more scarce and valued resources input are needed for schools to be 
more effective. A school is effective if it can acquire its needed resources. Therefore, 
inputs and acquisition of resources become the primary criteria of effectiveness 
(Etzioni, 1969; Yuchtman & Seashore, 1967). Quality of student intake, facilities, 
resources, and financial support procured from the central education authority, alumni, 
parents, sponsoring body or any outside agents are important indicators of 
effectiveness. This model is useful if the connections between inputs and outputs are 
clear (Cameron, 1984) and the resources are very limited for the school to achieve 
goals. Attraction of high quality student input is often assumed as a "necessary" 
condition for some schools to become effective or achieve high academic performance 
in public examinations. Obviously, this model has its defects because its over-emphasis 
on acquisition of inputs may reduce the school’s effort put to educational processes 
and outputs. The acquired resources may become wastage if they cannot be used 
efficiently to serve school functions. 



The Process Model 



O 
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From a system perspective, school inputs are converted into school 
performance and output through a transformational process in school. The nature and 
quality of school process often determine the quality of output and the degree to which 
the school goals can be achieved. Particularly in education, experience in the school 
process is often taken as a form of educational aims and outcomes. Therefore, the 
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process model assumes that a school is effective if its internal functioning is smooth 
and "healthy". Leadership, communication channels, participation, coordination, 
adaptability, planning, decision-making, social interactions, school climate, teaching 
methods, classroom management, and learning strategies are often used as indicators 
of effectiveness. School process in general includes management process, teaching 
process, and learning process. Thus the selection of indicators may be based on these 
processes, classified as management effectiveness indicators (e.g. leadership, decision- 
making), teaching effectiveness indicators (e.g. teaching efficacy, teaching methods), 
and learning effectiveness indicators ( e.g. learning attitudes, attendance rate). If there 
is a clear relationship between school process and educational outcomes, this model 
should be useful. To a certain extent, the current emphasis on leadership and school 
culture to school effectiveness may reflect the importance of the process model 
(Caldwell & Spinks, 1992; Cheng, 1994; Sergiovanni, 1984). The process model has 
its limitations, such as difficulty in monitoring processes and gathering related data, 
and the focus on means instead of ends (Cameron, 1978). 



The Satisfaction Model 



Recently, there is a strong emphasis on quality in school education. The 
concept of quality is in fact closely related to the satisfaction of clients’(or customers’, 
constituencies’) needs or the conformance to clients’ requirements and expectations ( 
Crosby, 1979; Tenner & Detoro, 1992). From this point, the recent demands for 
education quality reinforce the use of constituencies’ satisfaction in explaining and 
assessing school effectiveness. The satisfaction model defines that a school is effective 
if all its strategic constituencies are at least minimally satisfied. It assumes that the 
functioning and survival of a school are under the influence of its strategic 
constituencies (e.g. .principal, teachers, school management board, education authority, 
parents, students and the public), and school actions are mainly reactive to the 
demands of the strategic constituencies. Therefore, satisfaction of these demands is the 
basic criterion for school effectiveness (Keeley, 1984; Zammuto, 1982, 1984). If the 
demands of all the powerful constituencies are compatible and the school has to 
respond to these demands, this model may be useful in studying school effectiveness. 
The indicators of effectiveness are often the satisfaction of students, teachers, parents, 
administrators, education authority, school management committee, or alumni, etc. 

If the demands of powerful constituencies conflict and cannot be satisfied at the 
same time, the model may not be appropriate. Recently, due to the drastic society 
developments and serious international competition, more people become the strategic 
constituencies of schools and more parties have different interests in school functions. 
The expectations of these multiple powerful constituencies are often high, diverse, and 
even inconsistent. It is obviously difficult for schools to satisfy all these needs and 
expectations in a short time when the resources available for schools are so limited. 



The Legitimacy Model 




In current competitive education environment, schools have to compete seriously 
for resources and overcome internal barriers and face the external challenges and 
demands for accountability and “value for money” (Education & Manpower Branch 
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and Education Department, 1991; Education Commission, 1994 ). In order to gain the 
legitimacy for resources and survival, schools have to show evidence of accountability, 
satisfy the requirements of the community, and win the support of important 
constituencies. Along this line of thinking, the legitimate model suggests that a school 
is effective if it can survive as a result of engaging in legitimate or marketing activities. 
It assumes that schools "strive for legitimacy with the external public in order to 
enhance their longevity and avoid being selected out of the environment" (Cameron, 
1984, p.278). Therefore, the indicators of effectiveness are often related to the 
activities and achievements of public relations and marketing, accountability, school 
public image, reputation, or status in the community, etc. The model is useful when the 
survival and demise among schools must be assessed in a changing environment. The 
current emphasis on parental choice and accountability in educational reforms in both 
Western and Eastern societies seem to support the importance of the legitimacy model 
in assessing school effectiveness. Even though there is strong interest in using market 
mechanism and accountability system to enhance school effectiveness in current 
worldwide education reforms, the knowledge of their relationship to the full spectrum 
of school functions and effectiveness is still underdeveloped. Therefore, we should pay 
attention to the potential limitations of this legitimacy model when using it in 
educational reforms. 



The Ineffectiveness Model 

The model assumes that it is easier for the concerned school constituencies to 
identify and agree on criteria of school ineffectiveness than criteria of school 
effectiveness. Also identifying strategies for improving school effectiveness can be 
more precisely done by analyzing school ineffectiveness as opposed to school 
effectiveness. Therefore, this model is useful particularly when the criteria of school 
effectiveness are really unclear but the strategies for school improvement are needed. 
The indicators of ineffectiveness may include existing conflicts, problems, difficulties, 
defects, weaknesses, and poor performance. In general, many schools, particularly 
those new schools, are more concerned with overcoming obstacles to basic school 
effectiveness than pursuing excellent school performance. This model may be 
appropriate to them. For those practitioners such as school administrators and 
teachers, the ineffectiveness model may be more basic than the other models. It seems 
that “no ineffectiveness” may be the basic requirement for effectiveness. But if people 
are more interested in high school performance, this model is not sufficient. 

The Organizational Learnine Model 

Since the education environment is changing quickly, there seems no static factor 
or single practice that contributes to school effectiveness forever. Therefore, how to 
deal with the environmental impacts and internal process problems is the key question 
in determining a school’s effectiveness. The organizational learning model assumes that 
the impact of environmental changes and the existence of internal barriers to school 
functioning are inevitable and therefore, a school is effective if it can learn how to 
make improvement and adaptation to its environment. Whether the school and its 
members (particularly, the administrators and teachers) can leam to deal with the 
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change and reduce the internal hindrance is very important (Argyris, 1982; Argyris & 
Schon, 1978; Levitt & March, 1988; Louis, 1994; Lundberg, 1989). The line of 
thinking supports the current emphasis of strategic management and development 
planning in school (Dempster et al. 1993; Hargreaves & Hopkins, 1991). The model is 
particularly useful when schools are developing or involved in educational reform 
particularly in a changing external environment. The indicators of school effectiveness 
may include awareness of community needs and changes, internal process monitoring, 
program evaluation, environmental analysis, and development planning, etc. Obviously, 
the usefulness of this model will be limited if the connection between organizational 
learning process and school outcomes is not clear. For example, some old schools have 
a tradition of prestige that can attract high quality student input. Even though they may 
lack organizational learning, they can still win relatively high student achievement and 
high school status. 

The Total Quality Management Model 

Recently there is a rapidly growing emphasis on education quality (Cheng, 
1995a; Education Commission, 1994; Hughes, 1988). The concepts and practices of 
total quality management in schools are believed to be a powerful tool to enhance 
education quality and increase school effectiveness (Bradley, 1993; Cuttance, 1994; 
Greenwood & Gaunt, 1994; Murgatroyd & Colin, 1993). This model assumes that for 
long-term success, quality performance, or effectiveness, total management of the 
internal environment and process to meet the customers’ (or clients’, strategic 
constituencies’) needs is the key. The critical elements of total quality management in 
school include strategic constituencies’ (e g., parents, students, etc.) focus, continuous 
process improvement, and total involvement and empowerment of school members 
(Tenner & Detoro, 1992). According to the total management model, a school is 
effective if it can involve and empower all its members in school functioning, conduct 
continuous improvement in different aspects of the school process, and satisfy the 
requirements, needs, and expectations of the school’s external and internal powerful 
constituencies even in a changing environment. To a great extent, the total quality 
•management model of school effectiveness is an integration of the above models, 
particularly the organizational learning model, the satisfaction model, and the process 
model. According to the famous Malcolm Baldrige Award framework or the European 
Quality Award framework for total quality management, the key areas for assessing 
school effectiveness may include leadership, people management, process management, 
information and analysis, strategic quality planning, internal constituencies’ 
satisfaction, external constituencies’ satisfaction, operational results, students’ 
educational results, and impacts on society (Fisher, 1994; George, 1992) . Compared 
with other models, the total quality management model provides a more holistic or 
comprehensive perspective to understanding and managing school effectiveness. If the 
strategic constituencies’ needs and expectations are compatible and the technology and 
resources are available for such a full span of management in school, this model should 
be appropriate. 

As discussed above, each of the eight models has its own strengths and 
limitations. In different situations and different time frames, different models may be 
useful for monitoring and managing school effectiveness. Comparatively, the 
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organizational learning model and the total quality management model seem to be 
more promising for achievement of multiple school functions at different levels. 



Practical Possibilities and Dilemmas 

In addition to the above conceptual issues, there are also some important 
practical possibilities and dilemmas in implementation of monitoring school 
effectiveness. Specifically, the design of types of evaluation, purposes of monitoring, 
and choices of effectiveness indicators is a key issue in practicing monitoring school 
effectiveness. 

Functions of Effectiveness Indicators 



The design of effectiveness indicators should serve the function of assessing 
and monitoring school effectiveness at the site level and at the system level. According 
to Nuttall (1990), good indicators at the site level or system level can provide at least 
one of the following types of information: 

• Information that describes school performance at the site level and the 
system level in achieving desired educational conditions or results. 

• Information about features of individual schools / the schools system 
known to be linked with desired conditions and outcomes. The 
information helps school members, policy-makers, educators, and the 
public to predict future performance of the school/the schools system; 

• Information that describes enduring features of individual schools / the 
schools system. The information helps policy-makers and educators better 
understand how individual schools / the schools system works and to 
assess the implication of changes over time; and 

• Information that is relevant to education policy. This information 
provides insight into current or potential problems in the school/ the 
schools system that are of particular concern to policy-makers or that are 
susceptible to change through action. 

Specifically, the information provided by the indicators can be used to 
monitor school education at both the site level and the system level. At the school-site 
level, the information can be used to ensure school performance and effectiveness 
towards the planned school goals (functions) and education standards; can help the 
school to reflect school's environment, reconsider school's direction, re-establish 
policies, and re-plan action programs, and reorganize structures; can help 
individuals, groups, and the school to learn, improve, and develop; and can help the 
school to assure education quality and school accountability to the public, students, 
and parents. Similarly, the information can be used to ensure effectiveness of the 
schools system, re-plan and develop the schools system, and assure accountability 
of the schools system to the public. 

O 
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Basically, the above functions of effectiveness indicators may be classified as 
two types: 

• Formative Functions—for internal improvement and development and 

• Summative Functions— for accountability and assurance of education 
quality or cost-effectiveness. 



Practical Possibilities for Monitoring School Effectiveness 

From the above discussion, the assessment and monitoring of school 
effectiveness at the site-level or the system-level may involve the following basic 
considerations in practice: 

• The Choice of Indicators: What kinds of indicators will be used ? Input 
indicators? Process or performance indicators? Outcome indicators ? Or the 
combination of these indicators ? 

• The Purposes: For what functions the assessment and monitoring will be ? 
For internal improvement and development ? For accountability and quality 
assurance ? Or, for both of them? 

• The Agents/ Type of Evaluation: Who will be responsible for the assessment 
and monitoring ? School self-evaluation ( the school as the agent) ? External 
evaluation (by an independent agent) ? Or both of them ? 

Based on these considerations, a matrix (the choice of indicators x the 
purpose x the type of evaluation) may be proposed, as shown in Figure 2, to 
explore the practical possibilities in monitoring school effectiveness. This matrix 
illustrates that there are three major approaches to monitoring school effectiveness: 

The Cell Approach: It often uses only one cell or some separate cells of 
the matrix to formulate the assessment. For example, the assessment uses only 
school self-evaluation and input indicators, and serves the purpose of internal 
development ( i.e. cell A of the matrix). This approach is simple and easy and may 
be commonly used in school evaluation. But the results from it are often biased and 
fragmented because of the lack of complete information about the whole school 
process and the ignorance of other important aspects. Therefore the implications for 
improvement, development, and accountability are very limited. 

The Layer Approach: It uses one layer of the matrix to design the 
assessment and monitoring of education quality. Figure 3 provides two examples of 
the layers. The layer of example 1 suggests that the assessment of school 
effectiveness includes indicators of input, performance, and outcome, purposes for 
both internal development and accountability, and school self-evaluation but not 
external evaluation. The layer of example 2 proposes that the assessment covers 
school self-evaluation and external evaluation, serves purposes for both internal 
development and accountability, and uses performance indicators but not input and 
outcome indicators. The conception of assessment based on layer may be broader 
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than that on separate cell. The results and implications from this approach may not be 
so limited and biased as those from the cell approach. 

The Whole Matrix Approach: The cell approach and the layer approach 
have strong limitations due to the ignorance of some important components in the 
matrix. Therefore the comprehensive way to assess and monitor school effectiveness 
at the site level or the system level should be based on the whole matrix including all 
the components of indicators, purposes, and agents. The results and implications 
from the whole matrix approach should be more comprehensive, reliable, and useful. 
The current emphasis on the whole school review or inspection including school 
self-evaluation and external audit, serving formative and summative functions, and 
using multiple indicators may reflect the importance of a whole matrix approach to 
successful assessment and monitoring of education quality ( Queensland Department 
of Education, 1992; Education Commission, 1994; Scottish Office Education 
Department, 1991, 1992). Of course this approach may be more expensive and time 
consuming because it covers more components than the other two approaches. 

Under the constraints of limited resources, inevitably there are still some basic 
dilemmas and issues in the use of the whole matrix approach to monitoring school 
effectiveness. 



Practical Dilemmas in Monitoring School Effectiveness 

Based on the proposed matrix, some critical dilemmas and issues can be 
illustrated as follows: 

1. How to keep a balance between the two functions of monitoring school 
effectiveness? Even though we would like to ensure both school's internal 
development and accountability to the public through monitoring effectiveness, it is 
often difficult to maximize these two functions at the same time, particularly if the 
assessment is limited to one type of evaluation (say, self-evaluation) and one category 
of effectiveness indicator (say, input indicator). Over-emphasis on the accountability 
purpose often increases the tendency that the schools create self-defensive mechanism 
and subsequently this mechanism hinders school self learning and improvement. If we 
focus too much on school internal development and improvement but less on 
accountability assurance, those concerned may worry whether the schools are 
accountable and the educational services are worth the money invested. How to 
keep a balance between internal development and accountability in designing and 
implementing the monitoring of school effectiveness should be a critical issue. 
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Figure 2 

A Matrix of Monitoring School Effectiveness 
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2. How to keep a balance between school self-evaluation and external 
evaluation ? In general, external evaluation by an outside or independent agent is 
believed to be more objective for the accountability purpose. Traditionally, it is 
conducted by the school inspection team of the Education Department of the 
Government. Since the Education Department is often the dominant educational 
service provider through numerous schools, some people believe that it may not be 
in the right position to assess independently and objectively the quality of 
educational service provided in schools. 

Currently, there is a trend to establish an expert agent independent of the 
Education Department and schools to conduct external evaluation of the schools 
(Education Commission, 1994). No matter whether the external evaluation is 
conducted by the Education Department or an independent agent, it would involve 
a big number of inspectors and so would be very expensive if it is conducted in depth, 
covers a wide range of indicators, and samples an appropriately large number of 
schools. Furthermore, over-emphasis on external evaluation may limit the initiative 
of the schools and hinder their development. On the other hand, school self- 
evaluation is based on the schools' own initiative and resources and therefore less 
expensive. Also it is easier for the schools to be committed to it and can be better 
for internal improvement and development. Obviously, it may not be so convincing 
for accountability purpose because the information from self-evaluation may be 
biased by the evaluators themselves. To keep an appropriate balance between school 
self-evaluation and external evaluation is necessary in planning monitoring education 
quality. But how to do it? 

3. How to choose an appropriate combination of indicators ? For different 
purposes and types of evaluation, different kinds of indicators may be used. For the 
accountability purpose, outcome indicators such as academic achievements, absence 
of behavioral problems, attendance rate, etc. are often strongly emphasized in 
monitoring. For the purpose of improvement and development of the school, staff, and 
students, performance or process indicators such as systematic planning, school 
climate, classroom environment, resource management, leadership, motivation, 
teaching process, learning process, etc. would be more important. 

If we emphasize the external evaluation particularly for monitoring the whole 
school system, we tend to use the objective and quantitative indicators that are 
appropriate to nearly all the schools. But if we are concerned mainly with school 
self-evaluation, the set of indicators developed by one school may not be 
appropriate to the other. Even though people would like to include indicators as 
more comprehensive as possible in order to serve different purposes and types of 
evaluation, it is often difficult to include too many indicators because it will be veiy 
time-consuming and expensive particularly when the assessment involves many 
schools. Therefore, how to choose an appropriate combination of indicators of input, 
performance, and outcome to fit the needs of multiple purposes and different types 
of assessing and monitoring effectiveness inevitably becomes a critical question to be 
answered. 




22 



24 



Conceptual/Practical Possibilities 



4. Who to make decision on choosing effectiveness indicators and 
performance standards ? The choice of effectiveness indicators and performance 
standards may reflect the values, education goals, and preferences of those 
concerned. Unfortunately, there are often very few indicators and standards 
commonly accepted by all those concerned. To different people, the choice may be 
very different. For example, to policy makers, the indicators of resources inputs and 
policy-related outcomes may be more important because these indicators are related 
to allocation of resources, efficiency, and accountability. To educators or teachers, the 
critical indicators should include process indicators particularly those related to the 
quality of teaching and learning process. To parents, the academic achievement 
outcomes may be the most necessary indicators of effectiveness. 

The diversity of emphasis on indicators and standards may complicate and 
politicize the process of establishing a program of assessing and monitoring school 
effectiveness at the school-site level or at the system level. Based on different sets of 
indicators and standards, the directions and results of assessing and monitoring can 
be very different. Then, the implications to policy making, resources allocation, 
management responsibility, practice improvement, and accountability can be of great 
difference for all those concerned such as policy-makers, inspectors, school managers, 
principals, school administrators, teachers, parents, students and the community. 
Therefore, who are the right persons to make decision on the choice of indicators and 
standards is a controversial issue in planning assessment and monitoring of school 
effectiveness. 

For the purpose of accountability to the public, the policy-makers and the 
Education Department often dominate the decisions, particularly at the school system 
level. From the perspective of school-based management (Cheng, 1993b), 
members at the site-level should have their right and responsibility to decide 
indicators and standards for their own schools, in order to facilitate internal 
improvement and development. But how these indicators and standards can be 
consistent with those decided for the system level is often controversial. As a form 
of compromise or flexibility, the policy-makers may establish a set of core and basic 
indicators and standards that all schools have to use and observe. In addition to this 
set, the schools may decide other indicators and standards for their own purposes. 
Also, the decision making at the school level should include the participation of all 
the strategic constituencies such as teachers, parents, alumni, and students in 
addition to the school administrators. Currently the encouragement of participation in 
school-based management and school review reflects this line of thinking ( Cheng, 
1993b; David, 1989; Queensland Department of Education, 1992 ) 

5. How to manage the differences between the school-site level and the 
system level ? The assessment and monitoring of school effectiveness at the school- 
site level is often different from that at the school system level in terms of expected 
purpose, indicators, and types of evaluation to be used. The potential differences of 
assessing school effectiveness between the school-site level and the system level can be 
summarized as shown in Table 3. 

At the system level, the purpose of monitoring may be mainly at ensuring 
accountability and cost-effectiveness, most of the indicators to be used may be related 
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to inputs and outcomes, and the type of evaluation to be adopted may be based on 
external evaluation. It is often difficult to use too many indicators particularly those 
qualitative process indicators because it will be expensive and time consuming in a 
large sample of schools. Also monitoring at the whole school system may not include 
the information from school self evaluation in order to eliminate the potential bias in 
self report. The set of indicators and standards is often developed by the policy 
makers or external agent. The indicators are standardized and can be applied to all 
the schools in the school system. 

But at the school-site level, the purpose of monitoring may be focused on 
internal improvement and development, most of the indicators may be related to 
school process and outcomes, and the type of evaluation may be mainly school self- 
evaluation. The evaluation can cover a wide range of indicators including qualitative 
and quantitative indicators. The set of indicators and standards to be used is 
developed by schools themselves. The indicators are not necessarily standardized and 
can be varied across schools. The set of school-based indicators may include the core 
indicators and standards developed at the system level. 

How to manage the above potential differences between the school-site level 
and the school system level is important for effective assessing and monitoring of 
school effectiveness at these two levels. Some people may suggest to reduce the 
differences and make the two levels nearly the same by (1) converting the 
characteristics of monitoring at the site level into those at the school system level or 
the reverse; or (2) integrating the characteristics of both the school-site level and the 
school system level and making them appropriate to both these two levels. The line of 
thinking ignores the uniqueness of monitoring at each level and assumes conversion 
without any loss in monitoring function and integration without any additional cost 
and technical problems in assessment. Of course, this assumption is not true. For 
example, the conversion may emphasize mainly either internal development or 
accountability but not both. If using integration, it is often very difficult to keep a 
balance between the two basic functions of monitoring at each level and also it will be 
very expensive and time-consuming due to the inclusion of too many indicators, as 
discussed above. 

In order to serve the different purposes and fit the different conditions at these 
two levels, we should accept the potential differences and monitor school 
effectiveness at both levels. We can develop appropriate strategies to keep a balance 
between the levels and manage them to be mutually supplementary but not 
contradictory. 

Clarifying the above issues is necessary for the development of a monitoring 
school effectiveness system. To a great extent, the current school-based management 
movements such as "Victoria Schools of the Future" of Australia (Directorate of 
School Education, 1994) and "School Management Initiative" of Hong Kong 
(Education & Manpower Branch and Education Department, 1991) can contribute to 
solving the above issues of monitoring and enhancing school effectiveness. 
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Table 3 

Difference in Monitoring Effectiveness 
between the School-Site Level and the System Level 





The School-Site Level 


The School System Level 


Purpose of 
Monitoring: 


• Mainly for Internal 
Improvement & 
Development 


• Mainly for Accountability 
& Cost- Effectiveness 


Type of Evaluation: 


• Based on School Self- 
Evaluation by School 
Members 


• Based on External 
Evaluation by Outside 
Agent 


Nature of Indicators: 


• Mainly Process & Outcome 
Indicators 


• Mainly Input & Outcome 
Indicators 




• Both Qualitative & 
Quantitative 


• Mainly Quantitative 




• Both Objective & 
Subjective( Affective) 


• Mainly Objective 




• Fit for the School, Not 
Necessarily Standardized 


• Fit for all Schools in the 
System, Standardized 




• Developed by School 
Members, May include 
those Core Indicators at the 
System Level 


• Developed by Education 
Department or an Outside 
Agent 



Conclusion 

In the new century, schools have different explicit or implicit functions to serve 
the emerging needs at the individual, institutional, community, society, and 
international levels. The technical/ economic, human/social, political, cultural, and 
educational functions at five different levels suggest various conceptual possibilities as 
well as dilemmas in designing a framework for monitoring school effectiveness. Also 
the multi-models of managing school effectiveness at the school-level provide a 
spectrum of different conceptions and indicators for developing a framework of 
monitoring school effectiveness. The goal model, the resource-input model, the 
satisfaction model, the legitimacy model, the ineffectiveness model, the organizational 
model, and the total quality management model have their own strengths and 
weaknesses for evaluating and monitoring school effectiveness. If we are serious 
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about school effectiveness, the proposed conceptual possibilities and dilemmas 
suggest: 

(1) a paradigm shift in monitoring school effectiveness from the simplistic 
conception with focus mainly on the technical effectiveness at the individual 
level or institutional level or the goal model to a wide spectrum of multiple 
school effectiveness or multiple models; and 

(2) a careful consideration of conceptual dilemmas in handling the complicated 
relationships between different types of school effectiveness and using 
different models to conceptualize, evaluate, and manage school 
effectiveness at the school-site level. 



In practicing monitoring school effectiveness, the choice of indicators, the 
purpose of monitoring, and the type of evaluation may suggest different practical 
possibilities as well as dilemmas. How to maximize the advantages and possibilities 
and handle the existing dilemmas should be a major concern in designing and 
implementing a framework of monitoring school effectiveness. 

Hopefully, the considerations and implications proposed in this paper are 
useful to current educational reforms not only in Hong Kong but also in different parts 
of the world. 
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